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ABSTRACT 


This  is  the  Final  Report  for  Contract  N00014-77-C-0328,  covering 
a period  of  two  years  from  February  1977  to  February  1979.  The  last 
three  months  of  this  period  was  provided  through  a three-month  no-funds 
extension  to  the  original  contract.  Research  was  conducted  primarily 
in  two  areas  of  cognitive  strategies  for  on-the-job  training  (OJT). 

The  first  area  was  the  development  and  testing  of  a training  system  to 
improve  selectivity  in  text  processing  in  order  to  improve  performance 
during  OJT.  The  second  area  was  the  exploration  of  text  type  effects 
on  learning  from  text.  Preliminary  results  from  this  research  suggest 
that  learning  from  text  may  be  measurably  improved  through  the  applica- 
tion of  text  processing  techniques  appropriate  to  the  type  of  text  being 
read. 

In  addition  to  producing  computer  programs  for  training  in  selective 
text  processing,  the  BTL  staff  also  produced  four  technical  reports,  two 
chapters  in  books,  and  three  papers  for  professional  meetings. 


I 


I 


I 


» 


» 


t 


I . 


I-,  • ' 


!■ 


ii 


ACKNOWLEDGEMENTS 


t 


I 


I 


I 


I 


t 


C 


c 


r 


Joseph  W.  Rigney  conceived  of  the  research  program  described  in  this 
report.  His  death  on  September  25,  1978  was  a great  loss  to  all  who  had 
worked  with  him  and  to  scholars  in  his  fields  of  research.  We  at  Behavioral 
Technology  Laboratories  owe  great  debts--intellectLial , professional,  and 
personal— to  Joe  Rigney. 

The  support  and  encouragement  of  Marshall  Farr  and  Henry  Halff, 
Personnel  and  Training  Research  Programs,  Office  of  Naval  Research,  and 
Harry  O'Neil,  Jr.  and  Dexter  Fletcher,  Defense  Advanced  Research  Projects 
Agency,  are  gratefully  acknowledged. 

We  also  thank  Captain  James  R.  Mills,  Commanding  Officer  of  the  Naval 
Reserve  Officer  Training  Corps,  at  the  University  of  Southern  California, 
and  his  associates  Commander  Stoakes  and  Lieutenant  Swinburnson,  who 
assisted  both  in  the  selection  of  materials  for  use  with  our  experimental 
training  system  and  in  the  recruitment  of  N.R.O.T.C.  student  subjects. 

Other  Behavioral  Technology  Laboratory  staff  members  also  made  impor- 
tant contributions  to  the  work  of  this  contract,  including  Donald  Crook, 
Lynn  Gordon,  Kathy  A.  Lutz,  and  David  Werner. 


I 

j 

( 


1 


I 


r 


h 


e 


iii 


! 


TABLE  OF 


CONTENTS 


I INTRODUCTION  1 

II  AN  EXPERIMENTAL  SYSTEM  TO  IMPROVE  SELECTIVITY 

IN  TEXT  PROCESSING 4 

III  EFFECTS  OF  TEXT  TYPE  ON  LEARNING  FROM  TEXT  ....  8 

IV  RECOMMENDATIONS 13 

REFERENCES 14 


iv 


I.  INTRODUCTION 


I 


t 


I 


I 


I 


,i 

;i 

>1 

I 


I 

» 


This  report  covers  a period  of  two  years,  from  February  18,  1977  to 
February  28,  1979.  The  last  three  months  of  this  period  was  provided 
through  a three-month  no-funds  extension  of  the  contract.  The  research 
undertaken  was  motivated  by  a concern  over  the  ineffectiveness  of  technical 
documentation  for  communicating  the  information  used  by  technical  personnel 
to  maintain  and  repair  equipment  in  the  Navy.  This  problem  has  been 
attacked  by  other  researchers  by  investigating  the  effects  of  changes  in 
the  documentation.  Research  has  been  conducted  on  readability  of  manuals 
versus  reading  level  of  recruits,  improvements  in  manual  format,  the  use  of 
word  processing  systems,  and  compression  of  information  into  small  volumes 
(such  as  microfiche  or  holograms)  in  order  to  improve  storage  and  retrieval. 
The  approach  undertaken  in  this  contract  has  been  to  seek  means  for  improv- 
ing the  effectiveness  of  personnel  in  using  documentation.  Two  approaches 
have  been  pursued.  The  first  resulted  in  the  development  and  testing  of  a 
computer-based  training  system  to  improve  selectivity  in  text  processing. 

The  second  approach  was  to  explore  the  effects  of  different  types  of  texts 
on  readers'  memories  for  the  texts. 

During  the  period  of  the  contract,  four  technical  reports,  two  chapters 
in  books,  and  three  papers  for  professional  meetings  were  produced.  In 
addition,  computer  programs  for  teaching  selective  learning  techniques  were 
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developed  and  tested.  Evaluations  of  these  programs  were  conducted  with 
the  participation  of  Naval  Reserve  Officer  Training  Corps  students. 

The  progress  of  the  research  undertaken  for  this  contract  reflects  a 
growing  concern  for  the  importance  of  studying  the  basic  cognitive  proces- 
ses responsible  for  successful  learning  from  texts.  Although  initial 
efforts  to  develop  a computer-based  system  to  improve  self-directed  learn- 
ing from  text  met  with  moderate  success,  many  of  the  phenomena  observed 
during  the  experimental  evaluations  prompted  a concern  with  more  basic 
issues.  One  such  issue  is  whether  the  replacement  of  an  inefficient  but 
well-learned,  unconscious,  strategy  for  learning  with  a technically  more 
efficient  but  less  well-learned  strategy  will  actually  result  in  the  im- 
provement of  performance.  Well-learned,  "automatized"  processes  tend  to 
conflict  with  consciously  executed  strategies  in  some  contexts.  More  work 
needs  to  be  done  to  determine  under  what  circumstances  a conscious  strategy 
will  result  in  superior  performance  relative  to  an  inherently  inferior 
automatic  process.  In  addition,  research  is  called  for  to  determine  what 
variables  control  the  amount  of  training  required  for  the  automatization  of 
a learning  strategy  or  process. 

A second  issue  which  emerged  from  the  first  research  efforts  reported 
here  is  the  effect  of  text  type  on  learning  from  text.  Pilot  experiments 
and  informal  observations  suggested  that  there  were  important  differences 
among  texts  (beyond  the  usually  noted  differences  in  word  frequency  and 
syntactic  complexity)  that  could  affect  understanding  of  and  memory  for  the 
content  of  the  texts.  It  was  hypothesized  that  a psychological  variable 
called  text  type  could  be  associated  with  texts.  The  value  that  a given 
text  has  on  the  text  type  dimension  was  expected  to  affect  the  way  in  which 
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the  text  would  be  grouped  with  other  texts,  how  well  the  information  in  the 
text  would  be  remembered,  and  so  on.  A variety  of  experiments  were  conducted 
and  their  results  convincingly  support  these  hypotheses.  The  significance 
of  this  result  for  training  to  meet  job  performance  requirements  is  twofold. 
First,  it  is  possible  that  the  text  type  variable  may  be  manipulated  in  such 
a way  as  to  improve  readers'  memories  for  the  information  conveyed  by  a text. 
Second,  pilot  results  suggest  that  different  reading  strategies  may  be  differ- 
entially effective  at  promoting  memory  for  the  content  of  the  texts  of  differ- 
ent types.  Further  research  is  called  for  if  these  findings  are  to  be  applied 
to  improve  self-directed  learning  to  meet  job  performance  requirements. 
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Military  tasks,  such  as  troubleshooting  complex  electronics  equipment, 
often  require  the  use  of  texts,  such  as  technical  manuals.  In  many  cases, 
far  more  information  is  available  in  the  text  than  is  needed  for  the  accom- 
plishment of  the  task  at  hand.  The  purpose  of  this  research  was  to  develop 
a computer-based  instructional  system  to  teach  self-directed,  selective 
reading  skills.  The  approach  taken  was  to  develop  a computer  program  that 
provides  automated  aids  to  this  kind  of  self-directed  learning.  One  of  these 
functions  of  this  aids  system  was  to  promote  a careful  job  analysis,  including 
the  formulation  of  reading  objectives  relevant  to  the  job  task  at  hand. 

Another  function  was  to  permit  the  student/user  to  create  a task-specific  list 
of  portions  of  the  text,  and  to  require  that  these  be  related  by  the  student 
to  specific  objectives.  The  aids  system  also  maintained  a record  for  the  use 
of  the  student  of  his  or  her  progress  in  understanding  relevant  text  portions 
and  accomplishing  objectives.  The  structure  of  the  aids  system  was  designed 
to  promote  conceptually-driven  processing  in  the  use  of  texts  in  job-related 
tasks.  An  experimental  test  of  the  first  training  system  was  conducted  in 
October-November,  1977.  The  findings  of  this  study,  including  the  comments 
given  by  student  participants,  were  used  to  develop  a new  training  system.  The 
new  system  was  constructed  to  further  emphasize  conceptually-driven  aspects  of 
selective  text  processing,  and  new  memory  aids  were  provided  for  the  student. 

In  addition,  aspects  of  the  training  system  were  improved  on  pedagogical 
grounds.  An  experimental  test  of  the  new  system  wa*^  conducted  in  April -May,  1978. 
Students'  performances  were  evaluated  on  several  measures  of  selectivity  and  on 
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the  quality  of  their  planning,  as  well  as  on  the  overall  efficiency  of  their 
troubleshooting  performances.  In  addition,  the  student's  written  summaries 
of  their  self-reported  strategies  for  selective  text  processing  were  evalua- 
ted in  terms  of  a schema- theoretic  model  of  the  selective  text-processing 
skills  of  an  ideal  reader. 

The  results  of  these  experiments  suggest  that  readers  can  be  taught  to 
make  more  effective  use  of  texts  through  the  application  of  selective  proces- 
sing procedures.  However,  the  results  also  suggest  that  retraining  basic 
text  processing  techniques  may  be  a time-consuming  and  expensive  process, 
relative  to  the  amount  of  improvement  in  learning  from  texts  that  is  brought 
about.  Informal  observations  of  student  behavior  in  these  experiments  sug- 
gested that  the  lines  of  research  described  under  III,  below,  would  be  a more 
I effective  means  of  improving  self-directed  learning  from  text. 

Work  in  this  area  was  carried  out  from  February  of  1977  to  September  of 
1978.  Two  technical  reports,  two  chapters  in  books,  and  one  paper  prepared 
• for  a professional  meeting  describe  the  results  of  this  research. 
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Technical  Report  No.  84 


Allen  Munro,  Joseph  W.  Rigney,  and  Donald  E.  Crook  A formative  evaluation 
of  a computer-based  instructional  system  for  teaching  job-oriented 
reading  strategies.  January  1978. 


On-the-job  training  requires  considerable  independence  on  the  part  of  the 
trainee.  Unlike  a student  in  a classroom,  the  trainee  must  arrange  informa- 
tion resources  in  such  a way  that  he  can  learn  how  to  perform  his  specific 
task  without  wasting  valuable  time  reading  irrelevant  information.  He  must 
further  direct  this  learning  himself. 

A computer-based  aid  to  self-directed  learning  has  been  developed  to  meet 
this  need.  This  aids  system  is  implemented  on  the  PLATO  system  and  uses  the 
touch-panel  capability  of  the  PLATO-IV  terminal.  Students  are  presented  with 
a task  which  requires  complex  learning,  and  they  are  given  considerable  in- 
formation- -much  more  than  is  needed,  in  fact— to  attain  the  task.  The  aids 
system  is  designed  to  allow  students  to  break  down  their  task  into  a set  of 
more  easily  attained  objectives,  to  decide  when  information  is  relevant  to 
their  objectives,  and  in  general  to  monitor  their  progress  toward  achieving 
the  task. 

The  complete  training  aid  is  quite  complex,  so  that  students  are  trained  in 
its  use  over  a number  of  sessions.  New  features  of  the  system  are  introduced 
in  alternate  sessions,  and  students  then  practice  with  the  system  using  a new 
learning  task.  This  task  in  each  case  requires  the  student  to  troubleshoot 
or  debug  a simulated  device.  This  device  produces  output,  some  of  which  is 
defective,  and  the  student  is  required  to  locate  the  faulty  component  by  exam- 
ining the  defective  output  and  by  reading  an  on-line  "technical  manual"  for 
the  device. 

A pilot  experiment  has  been  completed  to  allow  a formative  evaluation  of  the 
self-directed  aids  system.  Although  the  results  of  this  experiment  found  no 
statistically  significant  differences  between  the  treatment  groups,  they  sug- 
gested directions  for  future  research. 


I 


Technical  Report  No.  88 


Donald  E.  Crook,  Allen  Munro,  Joseph  W.  Rigney,  and  Kathy  A.  Lutz 
A computer-based  training  system  for  selective  text  processing. 
August  1978. 


Self-directed  learning  is  that  type  of  learning  which  is  not  structured 
for  the  student  by  an  instructor.  Instead,  the  student  must  structure 
his  learning  himself  by  making  decisions  about  which  materials  are  rele- 
vant to  his  learning  goals,  which  materials  require  the  prior  understand- 
ing of  which  other  materials,  and  so  on.  A computer-based  system  has 
been  developed  to  train  students  in  this  type  of  learning. 

A revised  system  based  on  an  earlier  version  of  a computer-based  self- 
directed  learning  system  was  developed.  The  improved  system  described 
herein  contains  features  designed  to  make  it  easier  for  students  to  use. 

In  addition,  pedagogical  features  of  the  training  system  have  been  im- 
proved, to  give  students  an  opportunity  to  learn  the  system  completely. 

An  experimental  test  of  the  improved  system  was  designed  to  separate  out 
the  effects  of  training  in  self-directed,  learning  from  the  use  of  the 
system  itself.  Data  were  collected  on  four  different  measures  of  learning: 
effective  learning,  selective  learning,  planning,  and  verbal  report.  Re- 
sults of  the  experiment  found  that  there  were  no  significant  differences 
among  treatment  groups  in  the  performance  data  (the  first  three  learning 
measures),  even  though  one  of  the  experimental  groups  outperformed  the 
other  groups  in  every  measure.  On  the  measure  of  verbal  report,  however, 
this  experimental  group  performed  significantly  better  than  did  the  control 
group. 


III.  EFFECTS  OF  TEXT  TYPE  ON  LEARNING  FROM  TEXT 


Work  on  text  type  phenomena  was  conducted  from  October  of  1977  to  the 
end  of  the  contract.  The  results  of  the  research  are  described  in  two 
technical  reports  and  two  papers  delivered  at  professional  meetings. 

This  research  was  prompted  by  the  informal  observation  that  readers' 
memories  for  texts  seemed  to  be  determined  not  only  by  levels  of  lexical 
and  syntactic  complexity,  but  also  by  the  type  of  the  text.  Work  was  con- 
ducted on  the  structural  and  semantic  features  of  three  types  of  texts: 
single  narrative  stories,  definitional  explanations,  and  instructions.  A 
variety  of  characteristic  differences  on  both  semantic  and  structural  dimen- 
sions were  observed.  It  was  hypothesized  that  these  differences  would  result 
in  differences  in  the  amount  of  information  that  could  be  recalled  from  texts 
of  different  types.  Several  experiments  were  conducted  to  test  this  hypothe- 
sis, which  was,  in  general,  well  supported  by  the  data.  Analysis  of  students' 
recalls  also  revealed  that  the  extent  of  reordering  of  the  information  in 
texts  seemed  to  be  a function  of  text  type.  In  another  experiment,  it  was 
shown  that  text  type  is  a powerful  determiner  of  subjects'  responses  to  a 
sorting  task.  A clustering  analysis  of  text  sorting  data  bears  out  the  text 
type  assignments  proposed  for  particular  texts.  Taken  together,  these  results 
argue  for  the  psychological  validity  of  a text  type  variable. 

One  intriguing  and  unexpected  result  of  this  research  was  the  discovery 
that  different  text  processing  techniques  seem  to  be  differentially  effective 
for  texts  of  different  types.  Specifically,  it  was  found  that  when  students 
are  restricted  to  a single  exposure  to  texts,  they  recall  more  of  the  content 
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t of  Instructions  than  of  definitional  explanations.  However,  If  students  are 

required  to  reread  and  produce  written  sutmaries  of  the  texts,  they  recall 
more  of  the  content  of  definitions  than  of  instructions.  These  results 

^ suggest  that  memory  for  different  types  of  texts  may  be  Improved  through  the 

application  of  different  strategies— a different  strategy  may  be  ideal  for 
each  type  of  text.  Further  research  Is  called  for  to  establish  this  claim,  and, 

f If  results  warrant,  to  apply  the  findings  to  a system  for  Improving  learning 

from  texts  of  different  types. 
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Technical  Report  No.  85 


Uynn  Gordon,  Allen  Munro,  Joseph  W.  Rigney,  and  Kathy  A.  Lutz,  Summaries  and 
recalls  for  three  types  of  texts.  May  1978. 


SUMMARY 


A theoretical  orientation  for  the  study  of  different  types  of  texts  is  pre- 
sented. Schema  theory  is  proposed  as  a useful  metatheory  within  which  to 
develop  specific  theories  about  reading.  Both  theories  about  the  processes 
of  reading  and  theories  about  the  structure  of  what  is  read  can  be  readily 
formulated  in  schema  theory  terms.  It  is  proposed  that  readers  make  judg- 
ments about  the  types  of  texts  that  they  read  and  that  these  judgments  bring 
about  the  activation  of  expectations  with  respect  to  the  structure  and  mean- 
ing of  these  texts. 

Previous  work  on  the  structure  of  texts,  primarily  for  simple  narratives,  is 
reviewed.  Problems  with  earlier  formalisms  and  scoring  methods  are  discussed, 
and  heuristics  for  avoiding  these  problems  are  presented. 

Three  types  of  texts  were  selected  for  study.  One  type  was  the  simple  short 
story,  a type  closely  related  to  (and,  in  some  cases,  identical  with)  the 
kinds  of  texts  studied  by  other  researchers.  The  second  type  studied  were 
instructions.  The  third  type  was  definitional  explanations,  a type  well 
characterized  by  popular  science  articles.  Detailed  analyses  of  the  text 
structures  and  text  semantics  for  eight  texts  (three  stories,  two  instruc- 
tions, and  three  definitions)  are  presented.  Texts  of  the  different  types 
differ  from  each  other  in  consistent  ways  on  two  dimensions.  First  the  text 
structures  of  definitions  tend  to  be  organized  horizontally  rather  than  ver- 
tically, as  are  the  text  structures  of  stories  and  instructions.  Second, 
the  semantic  representations  of  stories  are  composed  of  specific  concepts, 
in  schema  theory  terms,  while  the  semantic  representations  of  instructions  and 
definitions  consist  primarily  of  generic  concepts.  On  the  basis  of  these 
differences  among  the  texts,  we  predicted  that  stories  would  be  better  remem- 
bered than  definitions.  Three  experiments  were  conducted  to  test  this  hypo- 
thesis. 

In  Experiment  One,  subjects  read  and  sumnarized  six  texts  and  later  recalled 
three  of  these  texts.  Analysis  of  the  sunnary  data  indicates  that  texts  of 
different  types  are  summarized  to  about  the  same  extent.  The  recall  data, 
however,  suggests  that  text  type  may  determine  the  amount  recalled.  Analysis 
of  the  recall  data  showed  that,  although  stories  were  remembered  best  (as  had 
been  predicted),  the  propositional  content  of  definitions  was  remembered 
better  than  that  of  instructions.  It  was  hypothesized  that  rereading  and 
summarizing  may  have  had  a differentially  facilitative  effect  for  later  re- 
call, benefiting  the  recall  of  definitions  more  than  instructions. 

In  order  to  test  this  hypothesis.  Experiments  Two  and  Three  were  performed. 
Subjects  heard  tape  recorded  texts  (in  Experiment  Two  the  same  set  of  texts 
used  in  Experiment  One;  in  Experiment  Three  a somewhat  different  set),  and, 
after  performing  a brief  interfering  task,  recalled  each  text  after  hearing 
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it.  They  were  therefore  not  able  to  reprocess  texts  as  they  had  been  able 
to  in  Experiment  One.  In  general,  the  results  of  these  experiments  confirmed 
our  predictions:  stories  were  recalled  better  than  instructions,  which,  in 
turn,  were  recalled  better  than  definitions.  Subjects'  recalls  in  these 
experiments  were  also  scored  for  the  amount  of  reordering  of  the  textual  ma- 
terial. This  analysis  showed  a very  powerful  effect  due  to  text  type.  Recalls 
of  definitions  showed  significantly  more  reordering  than  did  recalls  of  in- 
structions, which,  in  turn,  had  more  reordering  than  did  the  recalls  of  stories. 
These  results  are  also  in  accord  with  our  theory  that  stories  have  more  hierar- 
chical, differentiated  text  structures  than  do  instructions  or  definitions,  and 
that  definitions  have  less  hierarchical  structures  than  do  instructions. 

Subjects  in  these  two  experiments  were  also  requested  to  cluster  the  texts  in 
natural  groups  according  to  their  types,  as  they  perceived  them.  Their  group- 
ings were  remarkably  consistent  with  our  own  classifications. 

The  research  presented  demonstrates  the  need  for  a more  thorough  investigation 
both  of  the  nature  of  people's  expectations  for  differences  in  different  types 
of  text,  and  of  the  effects  of  such  expectations  on  understanding  and  memory. 
Further  research  is  also  needed  to  explore  the  hypothesis  that  texts  of  differ- 
ent types  may  benefit  differentially  from  the  application  of  particular  learn- 
ing strategies,  such  as  rereading  and  summarizing. 


Technical  Report  No.  91 


Allen  Munro,  Kathy  A.  Lutz,  and  Lynn  Gordon  On  the  psychological  reality 
of  text  types.  February  1979. 


SUMMARY 


Text  type  is  proposed  as  a psychologically  valid  construct.  Previous  re- 
search has  suggested  that  text  type  may  play  a role  in  a reader's  compre- 
hension of  and  memory  for  a text.  Two  experiments  were  conducted  to  ex- 
plore the  psychological  reality  of  text  types.  In  the  first  experiment, 
students  were  required  to  sort  twelve  texts  on  the  basis  of  their  similar- 
ities. The  resultant  sortings  were  subjected  to  a clustering  analysis. 
Despite  the  fact  that  other  bases  for  grouping  together  texts  existed— a 
number  of  pairs  of  semantically  related  texts  of  different  types  were  in- 
cluded—text  type  emerged  as  a powerful  determiner  of  group  membership. 

In  the  second  experiment,  students  listened  to  recorded  texts  and  then 
tried  to  recall  them.  As  was  predicted,  text  type  had  a significant  effect 
on  recall,  with  stories  being  recalled  more  fully  than  were  instructions  or 
definitions. 


IV.  RECOMMENDATIONS 


r 


Both  of  the  areas  of  research  pursued  under  this  contract  have  the 
potential  of  producing  products  useful  to  the  armed  services.  Further 
research  in  two  areas  is  called  for  before  such  products  can  be  developed, 
however.  The  first  of  these  areas  is  that  of  the  automization  of  learned 
behaviors.  Decisions  about  the  adoption  of  training  programs  designed  to 
replace  old,  well-learned,  inefficient  procedures  with  new,  more  efficient 
procedures  cannot  be  made  rationally  without  knowing  more  about  the  costs 
(in  time  and  training  effort)  of  making  the  new  strategy  an  effective  auto- 
matic response  in  the  trainee. 

The  second  area  of  research  suggested  by  the  findings  of  this  contract 
is  that  of  text  type  studies.  Much  of  the  psychological  research  currently 
being  conducted  on  reading  makes  extensive  or  even  exclusive  use  of  narra- 
tive text  stimuli.  The  findings  of  the  research  conducted  under  this  con- 
tract suggest  that  such  findings  may  not  be  applicable  to  the  cognitive  pro- 
cesses that  are  called  for  in  the  tasks  of  processing  more  technical  kinds  of 
texts,  such  as  are  most  often  used  in  military  jobs.  Further  basic  research 
is  called  for  to  determine  the  breadth  of  text  type  effects.  In  addition,  re- 
sults of  the  research  conducted  thus  far  suggest  potentially  profitable  direc- 
tions for  applied  research  on  type-specific  text  processing  strategies. 
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